Predictive processing of music and language in autism: Evidence from Mandarin and English speakers.

Atypical predictive processing has been associated with autism across multiple domains, based mainly on artificial antecedents and consequents. As structured sequences where expectations derive from implicit learning of combinatorial principles, language and music provide naturalistic stimuli for investigating predictive processing. In this study, we matched melodic and sentence stimuli in cloze probabilities and examined musical and linguistic prediction in Mandarin- (Experiment 1) and English-speaking (Experiment 2) autistic and non-autistic individuals using both production and perception tasks. In the production tasks, participants listened to unfinished melodies/sentences and then produced the final notes/words to complete these items. In the perception tasks, participants provided expectedness ratings of the completed melodies/sentences based on the most frequent notes/words in the norms. While Experiment 1 showed intact musical prediction but atypical linguistic prediction in autism in the Mandarin sample that demonstrated imbalanced musical training experience and receptive vocabulary skills between groups, the group difference disappeared in a more closely matched sample of English speakers in Experiment 2. These findings suggest the importance of taking an individual differences approach when investigating predictive processing in music and language in autism, as the difficulty in prediction in autism may not be due to generalized problems with prediction in any type of complex sequence processing.

Investigating implicit emotion processing in autism spectrum disorder across age groups: A cross-modal emotional priming study.

Cumulating evidence suggests that atypical emotion processing in autism may generalize across different stimulus domains. However, this evidence comes from studies examining explicit emotion recognition. It remains unclear whether domain-general atypicality also applies to implicit emotion processing in autism and its implication for real-world social communication. To investigate this, we employed a novel cross-modal emotional priming task to assess implicit emotion processing of spoken/sung words (primes) through their influence on subsequent emotional judgment of faces/face-like objects (targets). We assessed whether implicit emotional priming differed between 38 autistic and 38 neurotypical individuals across age groups as a function of prime and target type. Results indicated no overall group differences across age groups, prime types, and target types. However, differential, domain-specific developmental patterns emerged for the autism and neurotypical groups. For neurotypical individuals, speech but not song primed the emotional judgment of faces across ages. This speech-orienting tendency was not observed across ages in the autism group, as priming of speech on faces was not seen in autistic adults. These results outline the importance of the delicate weighting between speech- versus song-orientation in implicit emotion processing throughout development, providing more nuanced insights into the emotion processing profile of autistic individuals.

Exploring the role of singing, semantics, and amusia screening in speech-in-noise perception in musicians and non-musicians.

Sentence repetition has been the focus of extensive psycholinguistic research. The notion that music training can bolster speech perception in adverse auditory conditions has been met with mixed results. In this work, we sought to gauge the effect of babble noise on immediate repetition of spoken and sung phrases of varying semantic content (expository, narrative, and anomalous), initially in 100 English-speaking monolinguals with and without music training. The two cohorts also completed some non-musical cognitive tests and the Montreal Battery of Evaluation of Amusia (MBEA). When disregarding MBEA results, musicians were found to significantly outperform non-musicians in terms of overall repetition accuracy. Sung targets were recalled significantly better than spoken ones across groups in the presence of babble noise. Sung expository targets were recalled better than spoken expository ones, and semantically anomalous content was recalled more poorly in noise. Rerunning the analysis after eliminating thirteen participants who were diagnosed with amusia showed no significant group differences. This suggests that the notion of enhanced speech perception-in noise or otherwise-in musicians needs to be evaluated with caution. Musicianship aside, this study showed for the first time that sung targets presented in babble noise seem to be recalled better than spoken ones. We discuss the present design and the methodological approach of screening for amusia as factors which may partially account for some of the mixed results in the field.

The Relationship Between Autism and Pitch Perception is Modulated by Cognitive Abilities.

Previous studies reported mixed findings on autistic individuals' pitch perception relative to neurotypical (NT) individuals. We investigated whether this may be partly due to individual differences in cognitive abilities by comparing their performance on various pitch perception tasks on a large sample (n = 164) of autistic and NT children and adults. Our findings revealed that: (i) autistic individuals either showed similar or worse performance than NT individuals on the pitch tasks; (ii) cognitive abilities were associated with some pitch task performance; and (iii) cognitive abilities modulated the relationship between autism diagnosis and pitch perception on some tasks. Our findings highlight the importance of taking an individual differences approach to understand the strengths and weaknesses of pitch processing in autism.

The form and function processing of lexical tone and intonation in tone-language-speaking children with autism spectrum disorder.

Studies on how the form versus function aspect of tone and intonation is processed by autistic individuals have mainly focused on speakers of non-tonal languages (e.g., English) with equivocal results. While the samples' heterogeneous cognitive abilities may be contributing factors, the phenotype of tone and intonation processing in autism may also vary with one's language background. Thirty-eight cognitively able autistic and 32 non-autistic Mandarin-speaking children completed tone and intonation perception tasks, each containing a function and form condition. Results suggested that the abilities to discriminate tone and intonation were not impaired at either the form or function level in these autistic children, and that these abilities were positively associated with one another in both autistic and non-autistic groups. The more severe the autism symptoms, the worse the form- and function-level of tone and intonation processing. While enhanced tone and intonation processing has been found in a subgroup of autistic children, it may not be a general characteristic of the autistic population with long-term tone language experience. These findings reveal typical tone and intonation processing at both the form and function levels in cognitively able Mandarin-speaking autistic children and provide evidence for associated tone and intonation processing abilities across levels.

Explicit processing of melodic structure in congenital amusia can be improved by redescription-associate learning.

Congenital amusia is a neurodevelopmental disorder of musical processing. Previous research demonstrates that although explicit musical processing is impaired in congenital amusia, implicit musical processing can be intact. However, little is known about whether implicit knowledge could improve explicit musical processing in individuals with congenital amusia. To this end, we developed a training method utilizing redescription-associate learning, aiming at transferring implicit representations of perceptual states into explicit forms through verbal description and then establishing the associations between the perceptual states reported and responses via feedback, to investigate whether the explicit processing of melodic structure could be improved in individuals with congenital amusia. Sixteen amusics and 11 controls rated the degree of expectedness of melodies during EEG recording before and after training. In the interim, half of the amusics received nine training sessions on melodic structure, while the other half received no training. Results, based on effect size estimation, showed that at pretest, amusics but not controls failed to explicitly distinguish the regular from the irregular melodies and to exhibit an ERAN in response to the irregular endings. At posttest, trained but not untrained amusics performed as well as controls at both the behavioral and neural levels. At the 3-month follow-up, the training effects still maintained. These findings present novel electrophysiological evidence of neural plasticity in the amusic brain, suggesting that redescription-associate learning may be an effective method to remediate impaired explicit processes for individuals with other neurodevelopmental disorders who have intact implicit knowledge.

Emotion recognition in autism spectrum disorder across age groups: A cross-sectional investigation of various visual and auditory communicative domains.

Previous research on emotion processing in autism spectrum disorder (ASD) has predominantly focused on human faces and speech prosody, with little attention paid to other domains such as nonhuman faces and music. In addition, emotion processing in different domains was often examined in separate studies, making it challenging to evaluate whether emotion recognition difficulties in ASD generalize across domains and age cohorts. The present study investigated: (i) the recognition of basic emotions (angry, scared, happy, and sad) across four domains (human faces, face-like objects, speech prosody, and song) in 38 autistic and 38 neurotypical (NT) children, adolescents, and adults in a forced-choice labeling task, and (ii) the impact of pitch and visual processing profiles on this ability. Results showed similar recognition accuracy between the ASD and NT groups across age groups for all domains and emotion types, although processing speed was slower in the ASD compared to the NT group. Age-related differences were seen in both groups, which varied by emotion, domain, and performance index. Visual processing style was associated with facial emotion recognition speed and pitch perception ability with auditory emotion recognition in the NT group but not in the ASD group. These findings suggest that autistic individuals may employ different emotion processing strategies compared to NT individuals, and that emotion recognition difficulties as manifested by slower response times may result from a generalized, rather than a domain-specific underlying mechanism that governs emotion recognition processes across domains in ASD.

Mental representations of speech and musical pitch contours reveal a diversity of profiles in autism spectrum disorder.

LAY ABSTRACT: As a key auditory attribute of sounds, pitch is ubiquitous in our everyday listening experience involving language, music and environmental sounds. Given its critical role in auditory processing related to communication, numerous studies have investigated pitch processing in autism spectrum disorder. However, the findings have been mixed, reporting either enhanced, typical or impaired performance among autistic individuals. By investigating top-down comparisons of internal mental representations of pitch contours in speech and music, this study shows for the first time that, while autistic individuals exhibit diverse profiles of pitch processing compared to non-autistic individuals, their mental representations of pitch contours are typical across domains. These findings suggest that pitch-processing mechanisms are shared across domains in autism spectrum disorder and provide theoretical implications for using music to improve speech for those autistic individuals who have language problems.

The autistic brain can process local but not global emotion regularities in facial and musical sequences.

Whether autism spectrum disorder (ASD) is associated with a global processing deficit remains controversial. Global integration requires extraction of regularity across various timescales, yet little is known about how individuals with ASD process regularity at local (short timescale) versus global (long timescale) levels. To this end, we used event-related potentials to investigate whether individuals with ASD would show different neural responses to local (within trial) versus global (across trials) emotion regularities extracted from sequential facial expressions; and if so, whether this visual abnormality would generalize to the music (auditory) domain. Twenty individuals with ASD and 21 age- and IQ-matched individuals with typical development participated in this study. At an early processing stage, ASD participants exhibited preserved neural responses to violations of local emotion regularity for both faces and music. At a later stage, however, there was an absence of neural responses in ASD to violations of global emotion regularity for both faces and music. These findings suggest that the autistic brain responses to emotion regularity are modulated by the timescale of sequential stimuli, and provide insight into the neural mechanisms underlying emotional processing in ASD.

Perception and Production of Statement-Question Intonation in Autism Spectrum Disorder: A Developmental Investigation.

Prosody or "melody in speech" in autism spectrum disorder (ASD) is often perceived as atypical. This study examined perception and production of statements and questions in 84 children, adolescents and adults with and without ASD, as well as participants' pitch direction discrimination thresholds. The results suggested that the abilities to discriminate (in both speech and music conditions), identify, and imitate statement-question intonation were intact in individuals with ASD across age cohorts. Sensitivity to pitch direction predicted performance on intonation processing in both groups, who also exhibited similar developmental changes. These findings provide evidence for shared mechanisms in pitch processing between speech and music, as well as associations between low- and high-level pitch processing and between perception and production of pitch.

Individuals with autism spectrum disorder are impaired in absolute but not relative pitch and duration matching in speech and song imitation.

Individuals with autism spectrum disorder (ASD) often exhibit atypical imitation. However, few studies have identified clear quantitative characteristics of vocal imitation in ASD. This study investigated imitation of speech and song in English-speaking individuals with and without ASD and its modulation by age. Participants consisted of 25 autistic children and 19 autistic adults, who were compared to 25 children and 19 adults with typical development matched on age, gender, musical training, and cognitive abilities. The task required participants to imitate speech and song stimuli with varying pitch and duration patterns. Acoustic analyses of the imitation performance suggested that individuals with ASD were worse than controls on absolute pitch and duration matching for both speech and song imitation, although they performed as well as controls on relative pitch and duration matching. Furthermore, the two groups produced similar numbers of pitch contour, pitch interval-, and time errors. Across both groups, sung pitch was imitated more accurately than spoken pitch, whereas spoken duration was imitated more accurately than sung duration. Children imitated spoken pitch more accurately than adults when it came to speech stimuli, whereas age showed no significant relationship to song imitation. These results reveal a vocal imitation deficit across speech and music domains in ASD that is specific to absolute pitch and duration matching. This finding provides evidence for shared mechanisms between speech and song imitation, which involves independent implementation of relative versus absolute features. LAY SUMMARY: Individuals with autism spectrum disorder (ASD) often exhibit atypical imitation of actions and gestures. Characteristics of vocal imitation in ASD remain unclear. By comparing speech and song imitation, this study shows that individuals with ASD have a vocal imitative deficit that is specific to absolute pitch and duration matching, while performing as well as controls on relative pitch and duration matching, across speech and music domains.

The effects of background music on neural responses during reading comprehension.

The effects of background speech or noise on visually based cognitive tasks has been widely investigated; however, little is known about how the brain works during such cognitive tasks when music, having a powerful function of evoking emotions, is used as the background sound. The present study used event-related potentials to examine the effects of background music on neural responses during reading comprehension and their modulation by musical arousal. Thirty-nine postgraduates judged the correctness of sentences about world knowledge without or with background music (high-arousal music and low-arousal music). The participants' arousal levels were reported during the experiment. The results showed that the N400 effect, elicited by world knowledge violations versus correct controls, was significantly smaller for silence than those for high- and low-arousal music backgrounds, with no significant difference between the two musical backgrounds. This outcome might have occurred because the arousal levels of the participants were not affected by the high- and low-arousal music throughout the experiment. These findings suggest that background music affects neural responses during reading comprehension by increasing the difficulty of semantic integration, and thus extend the irrelevant sound effect to suggest that the neural processing of visually based cognitive tasks can also be affected by music.

The human brain processes hierarchical structures of meter and harmony differently: Evidence from musicians and nonmusicians.

The processing of temporal structure has been widely investigated, but evidence on how the brain processes temporal and nontemporal structures simultaneously is sparse. Using event-related potentials (ERPs), we examined how the brain responds to temporal (metric) and nontemporal (harmonic) structures in music simultaneously, and whether these processes are impacted by musical expertise. Fifteen musicians and 15 nonmusicians rated the degree of completeness of musical sequences with or without violations in metric or harmonic structures. In the single violation conditions, the ERP results showed that both musicians and nonmusicians exhibited an early right anterior negativity (ERAN) as well as an N5 to temporal violations ("when"), and only an N5-like response to nontemporal violations ("what"), which were consistent with the behavioral results. In the double violation condition, however, only the ERP results, but not the behavioral results, revealed a significant interaction between temporal and nontemporal violations at a later integrative stage, as manifested by an enlarged N5 effect compared to the single violation conditions. These findings provide the first evidence that the human brain uses different neural mechanisms in processing metric and harmonic structures in music, which may shed light on how the brain generates predictions for "what" and "when" events in the natural environment.

Impaired emotional processing of chords in congenital amusia: Electrophysiological and behavioral evidence.

This study investigated whether individuals with congenital amusia, a neurogenetic disorder of musical pitch perception, were able to process musical emotions in single chords either automatically or consciously. In Experiments 1 and 2, we used a cross-modal affective priming paradigm to elicit automatic emotional processing through ERPs, in which target facial expressions were preceded by either affectively congruent or incongruent chords with a stimulus onset asynchrony (SOA) of 200 msec. Results revealed automatic emotional processing of major/minor triads (Experiment 1) and consonant/dissonant chords (Experiment 2) in controls, who showed longer reaction times and increased N400 for incongruent than congruent trials, while amusics failed to exhibit such a priming effect at both behavioral and electrophysiological levels. In Experiment 3, we further examined conscious emotional evaluation of the same chords in amusia. Results showed that amusics were unable to consciously differentiate the emotions conveyed by major and minor chords and by consonant and dissonant chords, as compared with controls. These findings suggest the impairment in automatic and conscious emotional processing of music in amusia. The implications of these findings in relation to musical emotional processing are discussed.

Abnormal neural responses to harmonic syntactic structures in congenital amusia.

In music, harmonic syntactic structures are organized hierarchically through local and long-distance dependencies. This study investigated whether congenital amusia, a neurodevelopmental disorder of pitch perception, is associated with impaired processing of harmonic syntactic structures. For stimuli, we used harmonic sequences containing two phrases, where the first phrase ended with a half cadence and the second with an authentic cadence. In Experiment 1, we manipulated the ending chord of the authentic cadence to be either syntactically regular or irregular based on local dependencies. Sixteen amusics and 16 controls judged the expectedness of these chords while their EEG waveforms were recorded. In comparison to the regular endings, irregular endings elicited an ERAN, an N5, and a late positive component in controls but not in amusics, indicating that amusics were impaired in processing local syntactic dependencies. In Experiment 2, we manipulated the half cadence of the harmonic sequences to either adhere to or violate long-distance syntactic dependencies. In response to irregular harmonic sequences, an ERAN-like component and an N5 were elicited in controls but not in amusics, suggesting that amusics were impaired in processing long-distance syntactic dependencies. Furthermore, for controls, the neural processing of local and long-distance syntactic dependencies was correlated at the later integration stage but not at the early detection stage. These findings indicate that amusia is associated with impairment in the detection and integration of local and long-distance syntactic violations. The implications of these findings in terms of hierarchical music-syntactic processing are discussed.

The neural basis for understanding imitation-induced musical meaning: The role of the human mirror system.

Music can convey meanings by imitating phenomena of the extramusical world, and these imitation-induced musical meanings can be understood by listeners. Although the human mirror system (HMS) is implicated in imitation, little is known about the HMS's role in making sense of meaning that derives from musical imitation. To answer this question, we used fMRI to examine listeners' brain activities during the processing of imitation-induced musical meaning with a cross-modal semantic priming paradigm. Eleven normal individuals and 11 individuals with congenital amusia, a neurodevelopmental disorder of musical processing, participated in the experiment. Target pictures with either an upward or downward movement were primed by semantically congruent or incongruent melodic sequences characterized by the direction of pitch change (upward or downward). When contrasting the incongruent with the congruent condition between the two groups, we found greater activations in the left supramarginal gyrus/inferior parietal lobule and inferior frontal gyrus in normals but not in amusics. The implications of these findings in terms of the role of the HMS in understanding imitation-induced musical meaning are discussed.

Musical training modulates the early but not the late stage of rhythmic syntactic processing.

Syntactic processing is essential for musical understanding. Although the processing of harmonic syntax has been well studied, very little is known about the neural mechanisms underlying rhythmic syntactic processing. The present study investigated the neural processing of rhythmic syntax and whether and to what extent long-term musical training impacts such processing. Fourteen musicians and 14 nonmusicians listened to syntactic-regular or syntactic-irregular rhythmic sequences and judged the completeness of these sequences. Nonmusicians, as well as musicians, showed a P600 effect to syntactic-irregular endings, indicating that musical exposure and perceptual learning of music are sufficient to enable nonmusicians to process rhythmic syntax at the late stage. However, musicians, but not nonmusicians, also exhibited an early right anterior negativity (ERAN) response to syntactic-irregular endings, which suggests that musical training only modulates the early but not the late stage of rhythmic syntactic processing. These findings revealed for the first time the neural mechanisms underlying the processing of rhythmic syntax in music, which has important implications for theories of hierarchically organized music cognition and comparative studies of syntactic processing in music and language.

Sensitivity to musical emotion is influenced by tonal structure in congenital amusia.

Emotional communication in music depends on multiple attributes including psychoacoustic features and tonal system information, the latter of which is unique to music. The present study investigated whether congenital amusia, a lifelong disorder of musical processing, impacts sensitivity to musical emotion elicited by timbre and tonal system information. Twenty-six amusics and 26 matched controls made tension judgments on Western (familiar) and Indian (unfamiliar) melodies played on piano and sitar. Like controls, amusics used timbre cues to judge musical tension in Western and Indian melodies. While controls assigned significantly lower tension ratings to Western melodies compared to Indian melodies, thus showing a tonal familiarity effect on tension ratings, amusics provided comparable tension ratings for Western and Indian melodies on both timbres. Furthermore, amusics rated Western melodies as more tense compared to controls, as they relied less on tonality cues than controls in rating tension for Western melodies. The implications of these findings in terms of emotional responses to music are discussed.

Neural differences between the processing of musical meaning conveyed by direction of pitch change and natural music in congenital amusia.

Music is a unique communication system for human beings. Iconic musical meaning is one dimension of musical meaning, which emerges from musical information resembling sounds of objects, qualities of objects, or qualities of abstract concepts. The present study investigated whether congenital amusia, a disorder of musical pitch perception, impacts the processing of iconic musical meaning. With a cross-modal semantic priming paradigm, target images were primed by semantically congruent or incongruent musical excerpts, which were characterized by direction (upward or downward) of pitch change (Experiment 1), or were selected from natural music (Experiment 2). Twelve Mandarin-speaking amusics and 12 controls performed a recognition (implicit) and a semantic congruency judgment (explicit) task while their EEG waveforms were recorded. Unlike controls, amusics failed to elicit an N400 effect when musical meaning was represented by direction of pitch change, regardless of the nature of the tasks (implicit versus explicit). However, the N400 effect in response to musical meaning in natural musical excerpts was observed for both the groups in both types of tasks. These results indicate that amusics are able to process iconic musical meaning through multiple acoustic cues in natural musical excerpts, but not through the direction of pitch change. This is the first study to investigate the processing of musical meaning in congenital amusia, providing evidence in support of the "melodic contour deafness hypothesis" with regard to iconic musical meaning processing in this disorder.

Perception of hierarchical boundaries in music and its modulation by expertise.

Hierarchical structure with units of different timescales is a key feature of music. For the perception of such structures, the detection of each boundary is crucial. Here, using electroencephalography (EEG), we explore the perception of hierarchical boundaries in music, and test whether musical expertise modifies such processing. Musicians and non-musicians were presented with musical excerpts containing boundaries at three hierarchical levels, including section, phrase and period boundaries. Non-boundary was chosen as a baseline condition. Recordings from musicians showed CPS (closure positive shift) was evoked at all the three boundaries, and their amplitude increased as the hierarchical level became higher, which suggest that musicians could represent music events at different timescales in a hierarchical way. For non-musicians, the CPS was only elicited at the period boundary and undistinguishable negativities were induced at all the three boundaries. The results indicate that a different and less clear way was used by non-musicians in boundary perception. Our findings reveal, for the first time, an ERP correlate of perceiving hierarchical boundaries in music, and show that the phrasing ability could be enhanced by musical expertise.